快捷方式学习对深度学习模型很常见，但导致了退化的特征表示形式，因此危害了该模型的可推广性和解释性。但是，在广泛使用的视觉变压器框架中的快捷方式学习在很大程度上是未知的。同时，引入特定领域的知识是纠正捷径的主要方法，捷径为背景相关因素。例如，在医学成像领域中，放射科医生的眼睛凝视数据是一种有效的人类视觉先验知识，具有指导深度学习模型的巨大潜力，可以专注于有意义的前景区域。但是，获得眼睛凝视数据是时必的，劳动密集型的，有时甚至是不切实际的。在这项工作中，我们提出了一种新颖而有效的显着性视觉变压器（SGT）模型，以在没有眼神数据的情况下在VIT中纠正快捷方式学习。具体而言，采用计算视觉显着性模型来预测输入图像样本的显着性图。然后，显着图用于散布最有用的图像贴片。在拟议的中士中，图像贴片之间的自我注意力仅集中于蒸馏的信息。考虑到这种蒸馏操作可能会导致全局信息丢失，我们在最后一个编码器层中进一步介绍了一个残留的连接，该连接捕获了所有图像贴片中的自我注意力。四个独立公共数据集的实验结果表明，我们的SGT框架可以有效地学习和利用人类的先验知识，而无需眼睛凝视数据，并且比基线更好。同时，它成功地纠正了有害的快捷方式学习并显着提高了VIT模型的解释性，证明了传递人类先验知识在纠正快捷方式学习方面传递人类先验知识的承诺

直到最近，通过深度加强学习（DRL）已经实现了强大的机器人机器人。然而，为了高效地学习参数化的BipeDal行走，通常需要开发的参考资料，将性能限制为参考文献的性能。在本文中，我们建议设计用于从参考文献的模仿学习的自适应奖励功能。鼓励代理人模仿当其性能较低时的参考资料，同时在达到参考资料的限制时追求高性能。我们进一步证明，开发的参考资料可以通过在没有费力调整的情况下产生的低质量参考，并且只要他们能够提供先验的知识即可加快学习过程即可才能部署。

生成符合用户意图的可控视频是计算机愿景中的一种吸引人而具有挑战性的话题。为了依次启用可动性的控制，提出了一种新颖的视频生成任务，名为Text-Image-to-Video Generation（TI2V）。通过可控的外观和运动，TI2V旨在从静态图像和文本描述生成视频。 TI2V任务的关键挑战在于从不同方式的外观和运动方面既呈对齐，以及在文本描述中处理不确定性。为了解决这些挑战，我们提出了一种基于运动锚的视频发生器（MAGE），其具有创新的运动锚（MA）结构来存储外观运动对准表示。为了模拟不确定性并提高多样性，它进一步允许注入显式条件和隐式随机性。通过三维轴向变压器，MA与给定图像相互作用以递归地产生令人满意的可控性和多样性的下一个帧。伴随新任务，我们构建了基于MNIST的两个新的视频文本成对数据集，并满足了评估。在这些数据集上进行的实验验证了法师的有效性并显示了TI2V任务的吸引力。模型和数据集的源代码即将推出。

这项工作试图提供一种合理的理论框架，旨在从数据压缩和歧视性代表的原则解释现代深度（卷积）网络。我们认为，对于高维多类数据，最佳线性判别表示最大化整个数据集之间的编码率差和所有子集的平均值。我们表明，用于优化速率降低目标的基本迭代梯度上升方案自然地导致了一个名为Redunet的多层深网络，其共享现代深度网络的共同特征。深度分层架构，线性和非线性操作员，甚至网络的甚至参数都通过正向传播明确地构造了逐层构造，尽管它们通过背部传播可用于微调。所获得的“白盒”网络的所有组件都具有精确的优化，统计和几何解释。此外，当我们强制执行分类时，所以，所以网络的所有线性运算符自然地变为多通道卷曲。不变设置中的推导表明稀疏性和不变性之间的折衷，并且还表明这种深度卷积网络在光谱域中构建和学习的显着更有效。我们的初步模拟和实验清楚地验证了速率降低目标和相关的Redunet的有效性。所有代码和数据都可用于\ url {https://github.com/ma-lab-berkeley}。

本文回顾了关于压缩视频质量增强质量的第一个NTIRE挑战，重点是拟议的方法和结果。在此挑战中，采用了新的大型不同视频（LDV）数据集。挑战有三个曲目。Track 1和2的目标是增强HEVC在固定QP上压缩的视频，而Track 3旨在增强X265压缩的视频，以固定的位速率压缩。此外，轨道1和3的质量提高了提高保真度（PSNR）的目标，以及提高感知质量的2个目标。这三个曲目完全吸引了482个注册。在测试阶段，分别提交了12个团队，8支球队和11支球队，分别提交了轨道1、2和3的最终结果。拟议的方法和解决方案衡量视频质量增强的最先进。挑战的首页：https：//github.com/renyang-home/ntire21_venh

Learning with noisy-labels has become an important research topic in computer vision where state-of-the-art (SOTA) methods explore: 1) prediction disagreement with co-teaching strategy that updates two models when they disagree on the prediction of training samples; and 2) sample selection to divide the training set into clean and noisy sets based on small training loss. However, the quick convergence of co-teaching models to select the same clean subsets combined with relatively fast overfitting of noisy labels may induce the wrong selection of noisy label samples as clean, leading to an inevitable confirmation bias that damages accuracy. In this paper, we introduce our noisy-label learning approach, called Asymmetric Co-teaching (AsyCo), which introduces novel prediction disagreement that produces more consistent divergent results of the co-teaching models, and a new sample selection approach that does not require small-loss assumption to enable a better robustness to confirmation bias than previous methods. More specifically, the new prediction disagreement is achieved with the use of different training strategies, where one model is trained with multi-class learning and the other with multi-label learning. Also, the new sample selection is based on multi-view consensus, which uses the label views from training labels and model predictions to divide the training set into clean and noisy for training the multi-class model and to re-label the training samples with multiple top-ranked labels for training the multi-label model. Extensive experiments on synthetic and real-world noisy-label datasets show that AsyCo improves over current SOTA methods.

Recent years have witnessed significant growth of face alignment. Though dense facial landmark is highly demanded in various scenarios, e.g., cosmetic medicine and facial beautification, most works only consider sparse face alignment. To address this problem, we present a framework that can enrich landmark density by existing sparse landmark datasets, e.g., 300W with 68 points and WFLW with 98 points. Firstly, we observe that the local patches along each semantic contour are highly similar in appearance. Then, we propose a weakly-supervised idea of learning the refinement ability on original sparse landmarks and adapting this ability to enriched dense landmarks. Meanwhile, several operators are devised and organized together to implement the idea. Finally, the trained model is applied as a plug-and-play module to the existing face alignment networks. To evaluate our method, we manually label the dense landmarks on 300W testset. Our method yields state-of-the-art accuracy not only in newly-constructed dense 300W testset but also in the original sparse 300W and WFLW testsets without additional cost.

Exploring dense matching between the current frame and past frames for long-range context modeling, memory-based methods have demonstrated impressive results in video object segmentation (VOS) recently. Nevertheless, due to the lack of instance understanding ability, the above approaches are oftentimes brittle to large appearance variations or viewpoint changes resulted from the movement of objects and cameras. In this paper, we argue that instance understanding matters in VOS, and integrating it with memory-based matching can enjoy the synergy, which is intuitively sensible from the definition of VOS task, \ie, identifying and segmenting object instances within the video. Towards this goal, we present a two-branch network for VOS, where the query-based instance segmentation (IS) branch delves into the instance details of the current frame and the VOS branch performs spatial-temporal matching with the memory bank. We employ the well-learned object queries from IS branch to inject instance-specific information into the query key, with which the instance-augmented matching is further performed. In addition, we introduce a multi-path fusion block to effectively combine the memory readout with multi-scale features from the instance segmentation decoder, which incorporates high-resolution instance-aware features to produce final segmentation results. Our method achieves state-of-the-art performance on DAVIS 2016/2017 val (92.6% and 87.1%), DAVIS 2017 test-dev (82.8%), and YouTube-VOS 2018/2019 val (86.3% and 86.3%), outperforming alternative methods by clear margins.

The importance of learning rate (LR) schedules on network pruning has been observed in a few recent works. As an example, Frankle and Carbin (2019) highlighted that winning tickets (i.e., accuracy preserving subnetworks) can not be found without applying a LR warmup schedule and Renda, Frankle and Carbin (2020) demonstrated that rewinding the LR to its initial state at the end of each pruning cycle improves performance. In this paper, we go one step further by first providing a theoretical justification for the surprising effect of LR schedules. Next, we propose a LR schedule for network pruning called SILO, which stands for S-shaped Improved Learning rate Optimization. The advantages of SILO over existing state-of-the-art (SOTA) LR schedules are two-fold: (i) SILO has a strong theoretical motivation and dynamically adjusts the LR during pruning to improve generalization. Specifically, SILO increases the LR upper bound (max_lr) in an S-shape. This leads to an improvement of 2% - 4% in extensive experiments with various types of networks (e.g., Vision Transformers, ResNet) on popular datasets such as ImageNet, CIFAR-10/100. (ii) In addition to the strong theoretical motivation, SILO is empirically optimal in the sense of matching an Oracle, which exhaustively searches for the optimal value of max_lr via grid search. We find that SILO is able to precisely adjust the value of max_lr to be within the Oracle optimized interval, resulting in performance competitive with the Oracle with significantly lower complexity.

Recent deep learning methods have achieved promising results in image shadow removal. However, their restored images still suffer from unsatisfactory boundary artifacts, due to the lack of degradation prior embedding and the deficiency in modeling capacity. Our work addresses these issues by proposing a unified diffusion framework that integrates both the image and degradation priors for highly effective shadow removal. In detail, we first propose a shadow degradation model, which inspires us to build a novel unrolling diffusion model, dubbed ShandowDiffusion. It remarkably improves the model's capacity in shadow removal via progressively refining the desired output with both degradation prior and diffusive generative prior, which by nature can serve as a new strong baseline for image restoration. Furthermore, ShadowDiffusion progressively refines the estimated shadow mask as an auxiliary task of the diffusion generator, which leads to more accurate and robust shadow-free image generation. We conduct extensive experiments on three popular public datasets, including ISTD, ISTD+, and SRD, to validate our method's effectiveness. Compared to the state-of-the-art methods, our model achieves a significant improvement in terms of PSNR, increasing from 31.69dB to 34.73dB over SRD dataset.